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INTRODUCTION: 

Ultrasonic  transmission  tomography  holds  out  the  hope  of  being  a  discriminating  tool  for  breast  cancer 
screening  that  is  safe,  comfortable,  and  inexpensive.  From  its  inception,  however,  this  imaging 
modality  has  been  plagued  by  the  problem  of  how  to  quickly  and  inexpensively  obtain  the  data 
necessary  for  the  tomographic  reconstruction.  The  specific  aim  of  this  research  is  to  determine  how 
best  to  adapt  a  new  microfabricated  ultrasonic  sensor  (currently  under  development  for  defense 
applications)  into  a  breast  cancer  screening  tool.  The  sensor  converts  an  acoustic  wavefront  into  a 
modulated  optical  signal  over  an  entire  imaging  plane.  Using  this  device,  it  should  be  possible  to  obtain 
the  data  necessary  for  3D  imaging  of  a  breast  in  a  short  time,  without  ionizing  radiation,  and  without  the 
need  for  compression  of  the  breast.  In  the  first  year  the  research  focused  on  refinement  of  the  sensor 
design  and  development  of  reconstruction  algorithms.  In  the  second  year  the  research  focused  on 
enlarging  the  area  of  the  sensor,  improving  the  reconstruction  algorithms,  doing  phantom  studies,  and 
developing  physician-friendly  display. 


Final  Report  Body: 

In  this  report  we  detail  the  two  years  of  development  and  design  of  our  optically  parallel 
ultrasound  sensor  (OPUS).  The  tasks  that  were  addressed  in  the  first  year  were:  sensor 
refinement,  reconstruction  algorithm  development,  and  sensor  calibration.  The  second  year’s 
tasks  were  to  enlarge  the  sensor  aperture,  improve  the  reconstruction  algorithms,  use  the  system 
to  acquire  data  on  a  phantom,  and  investigate  physician-friendly  display  strategies. 


BACKGROUND: 


Scientific  Basis: 

The  basic  physical  principle  we  are  using  to  do  ultrasonic  sensing  is  frustrated  total  internal 
reflection  (a  consequence  of  optical  refraction)^ Refraction  occurs  when  a  wave  crosses  an 
interface  between  media  in  which  the  speeds  of  light  are  different  (i.e.  of  different  refractive 
indices)  [Snell's  Law].  If  light  moves  from  a  slow  medium  to  a  fast  one,  there  is  a  critical  angle, 
6c  =  sin'‘(ni/n2)  (where  n2  is  the  index  of  refraction  of  the  slow  medium  and  ni  is  the  index  in  the 
fast  medium),  beyond  which  the  light  is  totally  reflected.  This  simple  picture  ignores  the 
evanescent  wave,  which  extends  outside  the  high  index  medium,  falls  off  exponentially  in 
amplitude  to  almost  zero  within  one  wavelength  beyond  the  interface,  and  does  not  propagate. 
Frustrated  total  internal  reflection  occurs  when  another  slow  medium  intercepts  the  evanescent 
wave.  Some  light  tunnels  through  the  gap  and  propagates  into  that  medium.  The  amount  of  light 
that  tunnels  is  related  to  the  materials  involved,  the  polarization,  and  the  gap  width  (see  Figures  1 
and  2). 


Figure  1 .  (a)  Total  internal  reflection  (TIR)  occurring  at  the  interface  between  an  optically  slow  medium  (n2)  and  a  fast 
medium  (nl).  The  evanescent  wave  extends  into  the  fast  medium,  (b)  Frustrated  total  internal  reflection  occurs  when 
another  slow  medium  intercepts  the  evanescent  wave. 


Reflection  vs  Gap 


Gap  /  Wavelength 

Figure  2.  Reflection  vs.  gap  size  at  the  interface  between  slow  and  fast  media.  S  and  P  refer  to  optical  polarization. 
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Our  sensor  uses  frustrated  total  internal  reflection  to  make  an  incident  ultrasonic  wave  modulate  a 
pulsed  beam  of  light,  which  is  then  acquired  by  a  camera  and  computer.  A  sequence  of  images, 
each  taken  with  the  optical  pulse  at  a  different  source  acoustic  phase,  enables  us  to  reconstruct  the 
ultrasonic  phase  and  amplitude  over  an  entire  2-D  surface. 

Optics: 

We  exploit  frustrated  total  internal  reflection  by  building  an  array  of  acoustic  pixels.  Each 
acoustic  pixel  is  composed  of  a  thin  (0.1  micron)  silicon  nitride  membrane  suspended  on  short 
(0.2  micron)  gold  walls  over  an  optical  substrate.  The  gap  between  the  membrane  and  the  optical 
substrate  is  filled  with  air  (figure  3).  The  membrane  is  exposed  to  the  ultrasonic  couplant.  When 
an  ultrasonic  pressure  wave  travels  through  the  couplant  and  impinges  on  the  acoustic  pixel,  the 
membrane  deflects,  causing  a  change  in  the  amount  of  light  reflected  from  the  total  internal 
reflection  surface  of  the  acoustic  pixel.  An  array  of  acoustic  pixels  can  be  used  to  modulate  a 
beam  of  light. 


(a)  (b) 


Figure  3.  (a)  Top  view  of  an  array  of  acoustic  pixels,  (b)  Side  view  showing  acoustic  pixels  mounted  on  optical 
substrate 


Just  as  a  strobe  light  can  be  used  to  watch  the  vibration  of  a  drumhead,  we  are  using  a  strobed 
optical  source  to  watch  the  relative  phases  and  amplitudes  of  tens  of  thousands  of  acoustic  pixels, 
all  at  once.  Given  a  sequence  of  images  and  calibration  data,  we  can  extract  the  relative  phase 
and  amplitude  of  the  vibration,  and  thus  extract  the  relative  pressure  phase  and  amplitude  at  each 
acoustic  pixel.  To  obtain  this  information  we  illuminate  the  sensor  with  ten  sequences  of  optical 
pulses,  each  sequence  timed  to  act  as  a  strobe  light  at  a  specific  acoustic  phase  (figure  4). 


Phase  1 


Phase  2  Phase  3 


Period  Width  (50  ns) 

(0.5  ps) 


Phase  10 


Figure  4.  Ten  images  are  taken,  each  with  the  optical  pulse  at  a  different  position  in  the  acoustic  period. 
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We  extract  the  phase  and  amplitude  at  each  pixel  by  fitting  the  intensity  at  that  pixel  through  the 
sequence  to  the  form  I  =  B  +  A  sin(  2  7i  i  / 10  +  C  )  where  I  is  the  intensity,  B  is  the  background, 
A  is  the  amplitude  of  the  sinusoidal  variation,  i  is  the  index  of  the  image  in  the  sequence,  and  C  is 
the  phase  of  the  variation. 

The  optical  train  of  this  device  is  as  follows:  we  illuminate  the  sensor  using  an  LED,  the  light 
from  which  is  homogenized,  polarized,  and  collimated.  We  acquire  the  reflection  using  either  a 
CCD  still  camera  or  a  video  camera. 

Acoustics: 

The  imaging  technique  we  are  interested  in  is  transmission  ultrasound.  In  this  modality,  an 
acoustic  source  sends  out  a  pressure  wave  through  a  couplant,  such  as  water,  oil,  or  medical 
ultrasound  gel,  to  the  object  of  interest.  The  pressure  waves  are  transmitted  through  the  object, 
being  modified  in  amplitude  and  phase  along  the  way.  The  pressure  wave  emerges  from  the 
object  of  interest  and  travels,  via  the  couplant,  to  our  acoustic  sensor. 

Our  sensor  works  because  the  pressure  wave  flexes  the  membrane,  causing  it  to  vibrate  with  a 
phase  and  amplitude  that  are  functions  of  that  wave.  In  designing  our  sensor,  we  needed  to  have 
a  membrane  with  a  frequency  response  high  enough  to  vibrate  at  the  frequencies  of  interest  to  us 
(approximately  2  MHz). 

Design  Parameters: 

There  are  many  material  and  operational  parameters  that  must  be  chosen  correctly  in  order  to 
make  a  working  OPUS.  Among  these  are  the  membrane  and  wall  materials,  membrane  thickness, 
wall  height,  acoustic  pixel  size,  ultrasound  operating  frequency,  optical  source  characteristics, 
optical  pulse  characteristics,  and  camera  characteristics. 

Some  of  these  parameters  were  simple  to  choose,  such  as  the  material  for  the  membrane.  The 
fabrication  processes  available  to  us  restricted  our  choice  of  membrane  materials  to  silicon  and 
silicon  nitride.  As  we  wanted  to  do  this  work  with  visible  light,  we  chose  to  use  a  silicon  nitride 
membrane. 

Other  parameters  were  more  difficult  to  choose,  as  there  was  significant  interaction  between 
them.  For  instance,  the  membrane  thickness  and  the  acoustic  pixel  size  interact  with  each  other  to 
effect  the  sensitivity  and  frequency  response  of  the  sensor.  Thus,  we  relied  on  modeling  to 
narrow  the  ranges  of  these  parameters. 

Modeling: 

We  began  our  work  on  this  project  by  modeling  as  many  of  the  systems  and  processes  as 
possible.  We  used  an  in-house  reduced-dimension  finite-difference  time-domain  Maxwell's 
solver  (TSARLITE)^  to  model  the  optical  aspects  of  the  sensor  (FTIR),  a  3-D  mechanical 
dynamics  code  (DYNA3D)^^  to  model  the  acoustic  responses  of  the  membranes  and  their 
supports,  and  a  reduced-dimension  Beam  Propagation  Method  code  (BEEMER)^  to  model  the 
imaging  system  as  a  whole. 

Using  TSARLITE  we  were  able  to  determine  the  ranges  where  we  could  expect  FTIR  to  be 
useful,  and  bounded  the  permissible  thickness  of  the  membrane  and  the  heights  of  the  supports  it 
would  stand  upon.  This  modeling  was  used  to  optimize  the  optical  aspects  of  the  initial  design. 
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BEEMER  (originally  a  scalar  diffraction  optics  code)  was  used  to  examine  the  issues  that  arise 
when  the  sensor  is  used  for  diffraction  tomographic  imaging.  It  was  used  to  model  the 
tomographic  data  acquisition  process,  as  well  as  a  number  of  reconstruction  algorithms. 

The  simulation  program  used  most  heavily  was  DYNA3D.  We  used  this  program  to  model  our 
initial  sensor  design,  a  membrane  suspended  on  an  array  of  posts.  DYNA3D  showed  that  this 
design  had  neither  the  sensitivity  nor  the  frequency  response  necessary  to  allow  us  to  acquire  the 
data  we  required.  Guided  by  our  simulations,  we  developed  a  more  responsive  design,  a  set  of 
resonant  membranes  suspended  on  walls.  Simulation  showed  this  design  was  responsive  and 
sensitive,  but  had  problems  with  cross-talk  and  drift  of  the  resonant  frequency  as  a  function  of 
hydrostatic  pressure  (see  Figure  5).  Further  simulation  allowed  us  to  modify  the  design  by 
offsetting  the  cells  and  varying  their  size  so  as  to  greatly  reduce  cross-talk  (see  Figure  6). 


Resonant  Frequency  vs  Pressure 


Figure  5.  DYNA3D  simulation  led  to  these  curves  of  expected  resonant  frequency  with  respect  to  hydrostatic  pressure. 


Figure  6a 


Figure  6b 


Figure  6.  Simulation  showed  crosstalk  between  acoustic  pixels  to  be  a  problem.  Only  the  acoustic  pixel  in  the  upper 
left  is  stimulated.  Staggering  the  acoustic  pixels  as  in  (a)  reduces  the  crosstalk,  but  additionally  staggering  the  resonant 
frequencies  of  the  acoustic  pixels  as  in  (b)  reduces  the  crosstalk  significantly. 

Simulation  led  us  to  a  reasonable  first  design,  but  we  still  needed  to  do  experiments  to  verify  the 
simulations,  and  get  proof  of  principle  results.  To  do  these  experiments  we  built  a  test  system. 


Fabrication; 

The  membrane  and  its  supports  are  fabricated  as  depicted  in  Figure  7. 


Figure  7.  (a)  The  silicon  wafer  has  nitride  deposited  on  both  sides,  (b)  The  nitride  is  etched  away  on  one  side,  (c)  A 
layer  of  gold  is  deposited  on  the  nitride,  (d)  The  gold  is  etched  to  leave  support  walls,  (e)  The  silicon  is  etched  from  the 
back,  leaving  a  frame  that  supports  the  whole  system.  (0  The  membrane  and  walls  are  bonded  to  the  optical  substrate  , 
and  then  mounted  to  the  support  plate. 

Sensor  Parameters: 

As  a  result  of  proof-of-principle  experiments,  we  had  the  information  necessary  for  the  design 
and  fabrication  of  the  first  generation  sensor.  The  acoustic  pixel  size  is  70  microns  square,  with  a 
0.1  micron  thick  membrane,  mounted  on  walls  0.2  microns  high.  The  active  aperture  is  7  mm 
square.  The  sensor  is  currently  being  operated  at  a  frequency  of  2MHz,  at  acoustic  powers  of  20- 
200  mW/cm^ 

Sensor  Noise  Characteristics: 

The  CCD  camera  we  are  using  is  an  Apogee  Instruments  API.  The  images  it  provides  are  760  x 
510  pixels,  each  of  which  is  16  bits  deep.  We  expose  the  CCD  for  one  second  (two  million 
pulses).  We  are  using  a  HP  214B  pulse  generator  to  send  50  nanosecond  2.8  volt  pulses  to  a  red 
LED. 

To  determine  the  purely  optical  noise  characteristics  of  our  data,  we  collected  a  set  of  40  images 
with  no  acoustic  excitation.  The  noise  is  approximately  Gaussian  with  standard  deviations 


extending  from  24  to  60  counts.  In  taking  data,  with  the  acoustic  and  optical  parameters  as 
above,  we  collect  90  sequences  of  images  and  average  them,  reducing  the  noise  by  a  factor  of  9. 
Our  data  have  a  sinusoidal  variation  in  amplitude  of  30  to  100  counts. 

Data  Acquisition  System: 

The  data  acquisition  system  consists  of  an  OPUSensor,  a  tank  with  an  acoustic  source  and  object 
of  interest,  and  the  optics.  The  system  is  illustrated  in  schematic  form  in  figure  8.  A  Telulex  SG- 
lOOA  arbitrary  waveform  generator  (AWG)  provides  both  a  2  MHz  CW  signal  and  an  optical 
pulse  synchronization  signal.  This  waveform  generator  has  the  ability  to  generate  an  arbitrary 
waveform  (which  we  used  to  produce  a  2  MHz  sinewave  to  send  to  the  amplifier)  and  an  arbitrary 
sync  pulse  (which  we  used  to  drive  the  optical  pulse  generator)  simultaneously,  under  computer 
control.  The  2  MHz  sinusoid  is  amplified  and  fed  through  the  power  meter  to  the  acoustic  source 
in  the  water  tank.  The  sync  signal  goes  to  the  pulse  generator,  which  in  turn  excites  the  optical 
source  as  a  strobe  light.  The  pulse  generator  output  and  the  oscillator  output  are  both  fed  to  an 
oscilloscope  to  allow  the  user  to  place  the  optical  pulse  at  any  point  in  the  acoustic  phase, 
although  this  is  usually  under  computer  control.  The  optical  pulses  are  homogenized,  polarized, 
collimated,  and  fed  through  the  prism  to  the  sensing  surface,  and  the  reflected  light  is  captured  by 
the  camera  and  saved  in  the  computer.  The  user  has  control  over  the  oscillator  frequency,  the 
amplifier  gain,  the  camera  exposure  time,  the  optical  pulse  width,  amplitude,  and  placement  in  the 
acoustic  phase,  as  well  as  the  depth  of  the  water  in  the  tank. 

During  a  typical  experimental  run  the  water  tank  was  filled  to  the  desired  depth,  and  the  acoustic 
power  level,  camera  exposure  time,  optical  pulse  width,  and  optical  pulse  amplitude  were  set. 
The  object  of  interest  was  positioned  between  the  acoustic  source  and  the  sensor.  Sequences  of 
ten  images  were  acquired,  each  with  the  optical  pulse  occurring  at  a  different  acoustic  phase. 
Both  object  data  and  calibration  data  (without  the  target)  were  acquired. 
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Figure  8.  Schematic  view  of  the  data  acquisition  system.  The  arbitrary  waveform  generator  output  is  sent  to  the 
amplifier  (to  provide  the  acoustic  signal  to  the  transducer)  and  the  pulse  generator  (to  provide  the  optical  signal).  The 
optical  pulse  is  homogenized,  polarized,  collimated  and  sent  to  the  sensing  surface.  Upon  reflection  it  is  acquired  by 
the  computer  controlled  camera.  The  object  of  interest  is  optional. 
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TASKS: 

Task  1:  Sensor  Refinement. 

The  refinement  of  the  sensors  was  a  continuous  process  over  the  two  years  of  the  grant.  Sensor 
refinement  entailed  optimizing  sensitivity,  improving  manufacturing  techniques,  and  increasing 
the  speed  of  data  acquisition. 

Over  the  two  years,  we  experimented  with  modifications  to  the  sensor  membrane  to  improve  the 
sensitivity.  We  found  experimentally  that  the  resonances  of  the  acoustic  pixels  are  very  broad; 
therefore,  there  was  little  to  be  gained  by  adjusting  their  area  for  acoustic  optimality.  Instead  we 
increased  the  acoustic  pixel  size  slightly  so  as  to  increase  the  area  for  optical  illumination  and 
improve  our  optical  signal  to  noise  ratio. 

MEMS  Changes: 

We  have  made  multiple  design  changes  to  the  sensor: 

1.  We  changed  the  way  that  the  sensor  and  prism  are  assembled.  Previously,  sensors  were 
assembled  by  taking  a  porthole,  laying  the  sensor  on  top,  then  gluing  a  prism  over  the  top  (see 
Figure  9).  This  turned  out  to  be  a  suboptimal  assembly  technique,  often  leading  to  leaks 
around  the  membrane  frame  and  destroyed  sensors.  Now,  we  glue  the  membrane  frame  to 
the  plate  and  then  press  the  prism  against  the  membrane,  supporting  the  prism  from  the  rear 
with  a  frame  attached  to  the  plate  (see  Figure  10).  This  insures  that  the  membrane  frame  will 
not  leak,  and  it  allows  easier  pre-installation  testing  of  the  membranes  for  water-tightness. 


•  We  increased  the  robustness  of  the  membrane.  Originally,  our  membranes  were  fabricated 
under  high  tension  (500MPa).  By  altering  the  fabrication  parameters,  lower  tension 
(lOOMPa)  membranes  were  produced.  Prior  to  this  change,  membranes  would  rupture  at  the 
slightest  provocation.  With  the  lower  tension  membranes,  the  membranes  in  the  0.7  cm 
frames  last  indefinitely  (no  failures  since  the  change). 
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•  The  sensor  membrane  size  was  adjusted  from  1cm  on  a  side  to  0.7  cm  on  a  side.  Although 
this  reduced  the  aperture  size,  it  improved  the  sensor  fabrication  yield  and  robustness. 

•  We  began  the  process  of  scaling  up  the  sensor  aperture  by  using  paned  windows  of  the  .7  x  .7 
cm  membranes.  We  designed  and  fabricated  membranes  consisting  of  a  5  x  3  paned  array, 
with  at  least  a  90%  active  area.  However,  we  had  persistent  difficulty  with  the  larger  paned 
arrays.  The  silicon  wafers  available  to  the  MEM  and  the  electronic  industry  are  not  available 
optically  flat.  Over  a  4  cm.  wide  silicon  wafer,  the  surface  could  be  bowed  at  least  1  micron. 
This  does  not  seem  like  a  large  amount  of  distortion,  but  relative  to  the  0.2  micron  high  walls 
of  the  sensor,  it  is  a  large  deformity.  The  resultant  large  area  membranes  could  not  be 
assembled  into  sensors  because  they  developed  leaks  when  applied  to  the  optically  flat  prisms 
and  subjected  to  hydrostatic  pressure. 
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Figure  1 1  Larger  area  sensors  were  difficult  to  manufacture  because  of  unavailability  of  flat 

silicon  wafers. 

•  Because  of  the  persistent  problem  with  the  larger  area  sensors,  we  developed  a  completely 
new  concept  for  the  ultrasound  to  optical  transducer.  Since  the  basis  of  the  OPUS  design  is 
small  closed  air  cells  adjacent  to  a  prism,  we  deduced  that  a  plate  coated  with  a  transparent 
film  with  bubbles  should  be  able  to  function  in  the  same  capacity  as  the  OPUS  membranes. 
The  bubble-plate  that  we  developed  consisted  of  the  sensor  prism  coated  with  a  hydrophobic 
material  in  which  small  holes  had  been  made  (see  Figure  12).  The  idea  is  that  each  of  these 
holes  traps  and  retains  a  bubble.  The  bubble  retains  its  integrity  because  of  the  surface 
tension  of  the  water,  not  because  of  a  membrane,  so  it  should  be  more  sensitive  to  the 
ultrasonic  excitation,  as  well  as  having  a  higher  frequency  response.  We  spent  considerable 
effort  trying  to  engineer  this  concept,  and  built  several  prototypes.  However,  we  had  a 
persistent  problem  keeping  the  air  bubbles  in  the  film.  Our  first  attempts  involved  standard 
photoresists  and  lithography.  However,  since  the  aspect  ratio  (depth/width)  of  the  holes 
must  be  high  enough  to  trap  a  bubble,  and  a  coating  thickness  of  <0.2  microns  was 
necessary,  it  proved  impossible  to  get  small  enough  diameter  holes  by  this  process.  Next,  we 
used  a  LLNL  proprietary  process  to  make  0.2  micron  diameter  holes  in  a  Lexan  coating,  but 
the  resultant  plate  was  still  unable  to  maintain  air  bubbles. 
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Optical  Changes 

By  changing  from  our  Apogee  digital  camera  to  a  video  camera,  we  have  reduced  acquisition 
times  by  a  factor  of  approximately  100.  The  Apogee  camera  required  1  second  to  acquire  a  frame 
of  data  and  8  seconds  to  offload  it  into  the  computer.  We  are  able  to  acquire  video  images  at  a 
rate  of  10  frames  per  second.  This  has  significantly  changed  the  way  we  acquire  data. 
Previously,  we  set  the  optical  pulse  at  a  specific  acoustic  phase  for  each  image  acquired.  This 
added  additional  time  to  each  frame  acquired,  but  spread  long-term  drifts  in  the  data  acquisition 
system  parameters  over  all  phases.  This  was  important  because  it  could  take  3  hours  to  collect  a 
sequence  of  data.  Now  we  can  set  the  optical  pulse  in  the  acoustic  phase  and  acquire  all  the 
frames  we  wish  to  average  in  less  than  10  seconds.  Unfortunately,  the  video  camera  we  used  in 
the  first  year  was  noisier  than  the  Apogee  camera.  However,  by  acquiring  more  frames  we 
reduced  the  noise.  Ultimately,  we  purchased  a  high-speed  video  camera  with  excellent  signal  to 
noise  ratio.  Now  images  are  acquired  at  an  even  faster  rate,  and  the  fast  camera  enables  signal 
averaging  with  minimal  time  expenditure. 

In  switching  to  a  video  camera  we  have  qualitatively  changed  the  way  the  sensor  can  be  used. 
We  have  found  that  at  video  speeds  the  eye's  filtering  ability  enables  one  to  see  changes  in  the 
pressure  wavefronts  impinging  on  the  sensor  in  real  time.  By  changing  the  timing  of  the  optical 
pulse,  one  can  infer  the  shape  of  the  wavefront. 

Task  2:  Reconstruction  Algorithm  Development. 

As  a  crucial  step  in  verification  of  any  reconstruction  algorithm,  a  simulator  had  to  be  developed 
that  could  model  both  the  propagation  of  sound  through  a  phantom  and  reconstruction  of  images 
using  the  data.  We  built  the  simulator,  based  on  the  BEEMER  scalar  paraxial  wave  propagation 
code.  This  allowed  us  to  import  simulated  phantoms,  model  the  propagation  of  sound  through 
them,  and  experiment  with  reconstruction  algorithms.  We  used  a  linear  Rytov-approximation 
based,  filtered  backpropagation  algorithm^  to  do  the  2D  reconstruction  experiments  in  the  first 
year,  and  a  nonlinear  adjoint  method  algorithm  in  the  second  year. 

The  two  most  commonly  used  simplifications  in  linear  diffraction  tomography  are  the  Born  and 
Rytov  approximations.  The  Bom  approximation  assumes  that  there  are  no  large  differences  in 
the  index  of  refraction  anywhere  in  the  problem  space'*’^.  The  Rytov  approximation  assumes  that 
there  are  no  large  gradients  in  the  index  of  refraction®.  Both  assume  that  there  is  no  multiple 
scattering.  Devaney  first  published  the  filtered  backpropagation  algorithm  in  1982^.  Filtered 
backpropagation  uses  a  more  general  form  of  the  Fourier  projection-slice  theorem,  and  reduces  to 
filtered  backprojection  in  the  zero  wavelength  limit.  Whereas  the  Fourier  transform  of  a  zero 
wavelength  projection  in  real  space  equals  a  radial  slice  of  Fourier  space  perpendicular  to  the 
angle  of  the  projection,  the  Fourier  transform  of  a  finite  wavelength  projection  is  equal  to  a 
semicircle  in  Fourier  space  which  grazes  the  origin  at  an  angle  perpendicular  to  the  angle  of  the 
projection  (see  figure  13).  In  the  zero  wavelength  limit,  filtered  backprojection  is  a  fast  and 
simple  way  to  reconstruct  images  from  projections’.  In  the  case  of  finite  wavelength,  filtered 
backpropagation  is  an  attractive  reconstruction  method.  With  filtered  backprojection,  the 
projection  is  filtered  and  then  distributed  into  the  reconstruction  space  to  generate  an  image. 
Filtered  backpropagation  requires  that  the  filtered  projection  be  propagated  across  the 
reconstruction  space.  Propagation  is  a  more  computationally  intense  operation  than  simple 
projection. 

Based  on  the  experiments  we  performed  with  the  simulator,  we  built  a  stand-alone  2D 
reconstruction  code  which  used  a  simple  implementation  of  the  Devaney  Rytov  filtered 
backpropagation 
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Figure  13.  The  difference  between  zero  and  finite  wavelength  projections:  a)  zero  wavelength  limit,  Fourier  transforms 
of  projections  become  radial  lines,  b)  finite  wavelengths,  Fourier  transforms  of  projections  become  semicircles. 


During  the  second  year  of  the  project,  we  investigated  the  adjoint  method  of  nonlinear  iterative 
reconstruction^’^^.  In  a  nonlinear  reconstruction  algorithm,  a  calculation  is  made  of  what  data 
would  have  resulted  from  the  current  reconstruction  (forward  model).  The  output  of  the  forward 
model  is  compared  to  the  actual  experimental  data,  and  the  difference  is  used  to  generate  an 
update  to  the  reconstruction.  A  one  dimensional  optimization  is  performed  to  determine  how 
much  of  the  update  to  apply.  This  involves  many  iterations  of  the  forward  model  (which  is  why  it 
is  crucial  to  have  a  fast  model).  This  process  is  repeated  until  the  modeled  data  match  the 
experimental  data.  The  adjoint  method  is  a  fast  and  efficient  way  to  generate  the  update  to  the 
distribution  of  the  index  of  refraction  we  wish  to  determine. 

To  simplify  the  data  processing  we  assumed  that  there  was  insignificant  multiple  scattering  and 
that  large  deviations  (>10%)  in  sound  speed  do  not  occur^"^.  This  allows  us  to  use  a  paraxial  wave 
equation  solver  for  doing  the  forward  modeling.  A  more  correct  forward  model  would  be  a 
Helmholtz  equation  solver,  but  this  would  be  a  much  larger  computational  burden.  As  there  are 
no  published  derivations  for  the  paraxial  adjoint  method,  we  have  included  the  derivation  in 
Appendix  1.  The  final  result  of  the  derivation  is  that  the  current  update  to  the  index  should  be  a 
multiple  of  the  imaginary  part  of  \i/(r)\|/‘'(r)n(r),  where  \\f(r)  is  the  field  computed  with  the 
current  reconstruction,  \|/''(r)is  a  backpropagation  throught  the  current  reconstruction  with  the 
error  field  V|/  { r^bs )  “  Vexp  ( ^obs )  a  n(r)  is  the  current  reconstruction. 

Compared  to  the  linear  reconstruction  methods,  the  adjoint  method  yielded  far  more  accurate 
reconstructions  with  our  data,  although  at  a  much  higher  (by  a  factor  of  '-60)  computational  cost. 
However,  computational  costs  becomes  less  and  less  important  as  computer  speed  increases,  so 
the  accuracy  of  the  reconstruction  is  far  more  critical  than  the  speed  of  the  calculation. 
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Task  3:  Sensor  Calibration. 

Because  we  have  developed  experimental  protocols  that  include  normalizing  the  subject  data  to 
calibration  images,  optical  calibration  is  unnecessary.  Thus,  even  though  the  illumination  is  not 
uniform  over  the  whole  sensor,  the  calibration  images  normalize  out  the  nonuniformities. 

We  have  determined  that  the  acoustic  response  of  the  sensor  is  linear.  We  acquired  data  using  10 
different  input  voltage  levels  (O.OV  peak  to  peak  to  9.0V  peak  to  peak  to  the  amplifier)  (see  figure 
8).  We  collected  sequences  of  data  for  each  of  these  levels  and  extracted  from  them  the 
amplitudes  at  each  acoustic  pixel.  The  amplitude  vs.  power  level  at  each  pixel  is  linear,  with 
correlation  coefficient  of  0.984.  The  highly  linear  nature  of  the  data  indicates  that  the  deflections 
of  the  membrane  are  very  small  because  (as  can  be  seen  in  figure  2)  a  large  deflection  would  have 
excited  the  sensor  into  the  nonlinear  portion  of  the  optical  response  curve. 

By  measuring  the  power  input  to  the  source  transducer,  we  have  determined  the  minimum 
pressure  required  to  optically  observe  deflection  of  the  membrane.  The  membrane  can  detect 
pressure  changes  of  7kPa  (actually,  considering  the  efficiency  and  location  of  the  source 
transducer,  it  is  likely  seeing  at  least  a  factor  of  10  smaller  pressure). 

Task  4:  Phantom  Tank  Studies. 

Diffraction  Tomography  Experiments: 

For  a  test  of  diffraction  tomography  using  the  sensor  we  built  a  phantom  with  a  constant  2D  cross 
section.  The  phantom  consisted  of  three  pieces  of  0.5mm  diameter  fishing  line  placed  at  120 
degree  separations  around  a  1.5  mm  radius  circle  (see  figure  14).  These  materials  and 
configuration  were  selected  because  the  plastic  has  an  acoustic  index  not  too  different  from  the 
liquid  medium  (a  situation  that  mimics  the  variability  in  acoustic  index  in  the  breast),  and  the 
sizes  are  comparable  to  what  we  plan  to  image  with  the  system.  We  used  water  as  our  acoustic 
medium;  therefore,  an  acoustic  frequency  of  2  MHz  had  a  wavelength  of  0.74  mm.  Using  our 
simulator  (a  modification  of  the  BEEMER  modeling  code*)  we  determined  that  it  would  be 
possible  to  do  a  diffraction  tomographic  reconstruction  with  18  projections. 


Top  View 


Figure  14.  The  relationship  and  dimensions  of  the  phantom  and  the  sensor 


We  utilized  a  constant  cross-section  phantom  so  that  2D  reconstruction  (rather  than  3D)  could  be 
performed.  The  full  3D  reconstruction  algorithms  are  still  under  development. 
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Equipment  Modifications: 

We  mounted  a  rotation  stage  above  the  tank  and  suspended  the  phantom  3.5  mm  in  front  of  the 
sensor.  The  rotation  stage  was  manually  controlled.  We  mounted  the  source  transducer  far 
enough  away  (10  cm)  to  illuminate  the  phantom  with  a  planar  2MHz  ultrasound  wave. 

Data  Acquisition: 

The  data  necessary  for  reconstruction  of  the  phantom  are  a  set  of  18  projections,  where  each 
projection  consists  of  the  relative  acoustic  phase  and  amplitude  over  the  entire  sensor  surface. 
The  acoustic  period  was  divided  into  ten  discrete  phases,  as  sufficient  to  extract  the  sinusoidal 
parameters  at  each  acoustic  pixel.  A  set  of  10  images  is  a  sequence  from  which  phase  and 
amplitude  can  be  extracted.  We  took  90  sequences  that  were  then  summed,  to  increase  the  signal 
to  noise  ratio. 

The  data  for  each  projection  (18  projections  overall)  were  acquired  under  computer  control.  The 
data  consisted  of  90  sequences  of  10  images  (one  image  with  the  optical  pulse  at  each  of  10 
acoustic  phases).  To  acquire  each  image,  the  sinusoidal  waveform  was  downloaded  into  the 
arbitrary  waveform  generator  with  the  sync  pulse  at  the  appropriate  acoustic  phase,  the  camera 
shutter  was  opened  for  1  second,  then  the  data  was  downloaded  to  the  computer.  The  image  was 
summed  into  a  composite  image  for  the  appropriate  phase.  Upon  completion  of  the  acquisition  of 
the  90  sequences,  the  final  summed  images  were  saved  to  disk.  After  each  projection  was 
acquired,  the  phantom  was  rotated  20  degrees,  and  acquisition  was  repeated. 

After  the  projection  data  were  acquired,  we  obtained  calibration  data  by  removing  the  phantom 
and  averaging  90  sequences  of  10  phase  images. 


Data  Reduction: 

After  data  collection  we  had  18  projection  sequences  and  one  calibration  sequence,  each 
consisting  of  10  summed  images.  For  each  pixel  in  every  sequence,  the  intensity  was  fit  to  a 
sinusoid  with  phase  plus  a  background.  Since  the  illumination  over  the  optical  field  was  not 
uniform  (intentionally,  to  minimize  costs),  the  sinusoidal  amplitude  was  normalized  by  the 
background  intensity.  For  every  pixel  in  each  sequence  we  recorded  the  amplitude  ratio  and  the 
phase.  To  extract  relative  pressure  amplitude,  we  normalize  the  image  of  the  projection  ratios  by 
the  image  of  the  calibration  ratios,  on  a  pixel  by  pixel  basis.  This  yielded  18  images  of  relative 
pressure  amplitude  (see  figure  15).  By  subtracting  the  calibration  phase  from  the  target  phases, 
we  obtained  18  images  of  relative  phase. 


Figure  15.  The  18  images  of  relative  pressure  amplitude  obtained  by  the  OPUS  system  from  the  3 -monofilament 
phantom. 
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Figure  18.  3  Dimensional  rendering  of  the  reconstructed  OPUS-imaged  phantom  sketched  in  Figure  13. 
The  phantom  is  three  sections  of  monofilament  line,  each  0.5  mm  in  diameter.  This  reconstruction  was 
accomplished  with  the  Devaney  Rytov  filtered  backpropagation  method. 


Task  5:  Physician-friendly  Display 

There  are  several  important  issues  and  principles  involved  in  display  of  breast  ultrasound  data  for 
visual  analysis. 

The  information  must  be  viewed  on  both  the  macro  and  the  micro  level  for  detection  of  abnormal 
areas.  To  that  end,  the  data  need  to  be  viewed  first  in  a  non-obtmsive  grid  for  orientation,  in  a 
position  similar  to  what  would  be  seen  in  a  standing  person.  A  faint  line  will  indicate  the  skin 
surface  and  the  deepest  scanning  margin  (See  Figure  19)  Grey  will  be  used  for  the  background 
grid  and  for  the  deep  margin  lines,  so  as  to  give  orientation  without  distracting  from  the 
overlying  data  (See  Figure  20).  Black  outlines  are  to  be  avoided,  as  they  draw  the  eye  to  the 
unimportant  delineators,  rather  than  to  the  displayed  data'^. 

The  colors  for  sound  speed  and  attenuation  will  be  in  yellows  and  blues,  so  as  to  be  both  visually 
comfortable  and  to  maximize  visual  contrast  for  persons  with  red/green  color  blindness.  The 
colors  will  be  translucent  to  enable  visualization  of  data  through  the  entire  breast  at  once  (See 
Figure  21).  Varying  color  saturation  will  indicate  the  scale.  It  may  be  desirable  to  filter  values 
for  sound  speed  and  attenuation  to  display  value  ranges  of  greatest  interest,  avoiding  overly  dense 
display.  However,  it  will  be  necessary  to  acquire  data  on  an  entire  breast  before  it  is  possible  to 
determine  details  of  optimal  display. 

After  initial  survey  of  the  breast,  the  data  can  be  rotated  about  all  axes  or  sliced  in  arbitrary  planes 
to  facilitate  viewing.  Specific  volumes  can  be  extracted  for  viewing  without  the  surrounding 
tissue. 


p.l8 


Deep 
Margin 
Of  Scan 


Skin  surface 


Figure  19.  Representation  of  display  technique  for  sound  speed,  shown  in  yellow,  with  color  saturation 
indicating  the  scale.  The  skin  surface  and  deepest  margin  are  outlined  in  grey.  This  is  a  lateral  view. 
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Figure  20.  Antero-posterior  representation  of  display  technique  for  sound  speed,  shown  in  yellow,  with 
color  saturation  indicating  the  scale.  The  deepest  margin  is  outlined  in  grey.  Note  that  the  grid  scale  is  in 
an  even  lighter  grey  than  the  depth  outline,  so  as  to  give  reference  but  not  intrude  on  the  data. 
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Figure  21.  Antero-posterior  representation  of  display  technique  for  sound  speed,  shown  in  yellow,  with 
color  saturation  indicating  the  scale.  Attenuation  is  indicated  in  blue,  with  saturation  indicating  the  scale. 
The  deepest  margin  is  outlined  in  grey 


SUMMARY: 

We  have  developed  a  new  type  of  ultrasound  sensor  that  makes  it  possible  to  sense  the  pressure 
field  over  a  plane.  The  sensor  uses  the  phenomenon  of  frustrated  total  internal  reflection  to 
modulate  the  reflection  of  an  optical  beam  depending  on  the  deflection  of  a  0.1  micron  thick 
silicon  nitride  membrane  that  covers  an  acoustic  pixel.  Our  acoustic  pixels  are  defined  by  gold 
walls  0.2  microns  tall  and  enclosing  a  square  air  space  approximately  70  microns  on  a  side.  We 
have  used  an  array  of  10,000  acoustic  pixels  to  acquire  data  yielding  diffraction  tomographic 
images  of  a  3-monofilament  phantom. 

We  have  improved  the  sensor  speed  and  robustness.  We  have  implemented  two  different  types  of 
2D  reconstruction  algorithms,  based  on  filtered  backpropagation  and  adjoint  methods  of  solution. 
We  have  developed  acquisition  protocols  that  obviate  the  need  for  optical  calibration,  and  have 
performed  an  acoustical  calibration.  We  constructed  an  ultrasound  tank  with  a  rotation  stage,  and 
we  acquired  data  from  a  3D  phantom.  The  modeling  simulations  of  the  data  and  the  actual  data 
from  the  phantom  were  in  good  agreement. 

We  have  explored  techniques  of  physician-friendly  display,  and  developed  an  initial  approach  for 
data  display  from  patients. 
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KEY  RESEARCH  ACCOMPLISHMENTS: 


Sensor  Refinement 

•  We  changed  the  fabrication  process  to  generate  low-tension  membranes  in  0.7  cm  square- 
frames.  These  proved  far  more  durable  than  high-tension  membranes  in  1.0  cm  square- 
frames. 

•  We  determined  that  an  acoustic  pixel  size  of  70  microns  is  appropriate  for  excitation 
frequencies  of  up  to  2  MHz. 

•  We  incorporated  a  video-rate  camera  for  data  acquisition.  It  not  only  increased  the  speed  of 
acquisition  by  a  factor  of  100  but  also  made  the  system  more  like  a  fluoroscope  in  that  one 
could  watch  changes  in  the  acoustic  pressure  waveform  in  real-time. 

Reconstruction  Algorithm  Development 

•  We  discovered  that  our  sensor  can  collect  data  with  a  high  enough  signal  to  noise  ratio  that 
they  can  be  reconstructed  into  an  image. 

•  We  determined  that  the  Rytoff  approximation  is  sufficient  for  reconstructing  our  phantom 
data. 

•  We  developed  filtered  backpropagation  and  paraxial  adjoint  reconstruction  codes. 

Sensor  Calibration: 

•  We  implemented  data  acquisition  protocols  such  that  no  prior  optical  calibration  is  necessary 
because  optical  calibration  data  are  acquired  as  a  part  of  normal  data  acquisition. 

•  We  determined  that  the  sensor  is  linear  in  its  response  to  acoustic  excitation. 

•  We  demonstrated  that  the  sensor  can  detect  pressure  amplitudes  below  7  kPa. 

Phantom  Tank  Studies 

•  We  modified  the  test  tank  to  be  used  for  phantom  studies. 

•  We  acquired  data  from  a  simple  phantom. 

•  We  reconstructed  images  using  linear  diffraction  tomography  and  adjoint  methods  of 
solution. 

Physician-friendly  Display 

•  We  researched  the  graphics  literature  and  adopted  accepted  techniques  for  optimal  data 
display. 

REPORTABLE  OUTCOMES: 

We  have  submitted  a  manuscript  to  Journal  of  the  Acoustical  Society  of  America  for  publication. 

CONCLUSIONS: 

We  have  developed  a  new  type  of  ultrasound  sensor  that  makes  it  possible  to  sense  the  pressure 
field  over  a  plane.  The  sensor  uses  the  phenomenon  of  frustrated  total  internal  reflection  to 
modulate  the  reflection  of  an  optical  beam  depending  on  the  deflection  of  a  0.1  micron  thick 
silicon  nitride  membrane  that  covers  an  acoustic  pixel.  Our  acoustic  pixels  are  defined  by  gold 
walls  0.2  microns  tall  and  enclosing  a  square  air  space  approximately  70  microns  on  a  side.  We 
have  used  an  array  of  10,000  acoustic  pixels  to  acquire  data  yielding  diffraction  tomographic 
images  of  a  3-monofilament  phantom. 

We  have  improved  the  sensor  speed  and  robustness.  We  have  implemented  2D  reconstruction 
algorithms.  We  have  developed  acquisition  protocols  that  obviate  the  need  for  optical  calibration, 
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and  have  performed  an  acoustical  calibration.  We  built  a  phantom  tank  with  rotation  stage  and 
imaged  a  phantom. 

The  development  of  this  technology  is  important  for  the  early  detection  of  breast  cancer.  It  offers 
a  potential  way  of  implementing  volumetric  imaging  of  the  breast  without  the  ionizing  radiation 
and  discomfort  of  x-ray  mammography  or  the  cost  of  MRI.  To  fulfill  this  promise,  we  must  scale 
up  the  sensor  and  develop  further  3D  reconstruction  algorithms. 
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APPENDIX 

Paraxial  Adjoint  Method  Frdchet  Derivative  Derivation 
(following  the  method  of  Norton’®) 

* 

The  paraxial  wave  equation  arises  from  the  wave  equation: 

VV-4^=o 

c  dt^ 

where  Y  is  the  velocity  potential,  c  is  the  nominal  speed  of  sound,  and  n  is  the 
acoustic  index  of  refraction.  If  we  assume  the  following  form  for  W  : 

(where  co  is  the  frequency  of  interest,  and  k  is  the  wave  number  in  the  z  direction)  the 
wave  equation  reduces  to  the  Helmholtz  equation 


c  dz 


where  w  Ic  ,  and  V  is  the  part  of  the  Laplacian  perpendicular  to  z.  Note 
that  up  until  now  we  have  been  dealing  with  the  velocity  potential.  The  perterbation  in 

the  pressure  p  due  to  the  velocity  potential  is  given  by  p=p -  ,  where  p  is 

dt 

the  density  of  the  acoustic  medium.  However,  given  the  form  of  Y  ,  we  can  take,  the 
derivative  of  the  Helmholtz  equation  with  respect  to  time,  multiply  by  / p/to  ,  and  then 
Y  will  be  isomorphic  with  p  . 

Looking  more  closely  at  the  partial  derivative  with  respect  to  z  we  find 


d  Y  rd^ip  dip  1 

— Y+2jk- - k  Y]e 

oz  dz  dz 


-tiwt-kz) 


If  we  assume  the  first  term  on  the  right  hand  side  is  insignificant  compared  to  the  other 
two,  insert  the  other  two  terms  into  the  wave  equation,  and  divide  through  by  the 
exponential  term  and  2ik  ,  we  obtain  the  paraxial  wave  equation: 

We  want  to  find  the  index  distribution  n (7)  that  most  closely  matches  that  which 
produced  the  phase  and  amplitude  at  the  observation  points  of  the  problem.  This 
distribution  will  minimize  the  weighted  squared  error  functional: 

E[n{r)]=^'^  wjYirJ-YoJOf  (2) 

where  M  is  the  number  of  observation  points,  is  the  weight  of  the  rri^  point, 
(//  is  the  solution  to  the  paraxial  wave  equation  with  the  current  estimate  of  the  index 
distribution,  and  the  are  the  scattering  data. 

We  want  to  find  the  n(r)  that  minimizes  the  error.  Represent  the  variation  of  the 
error  with  a  change  in  index  distribution  this  way: 
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5E[n{r)]=fjj'V  pE{r)6n{r)d^r 

space  ' 

where  5  is  the  variation  operator,  and  V ^E  is  the  Fr^chet  derivative  (the  quantity  we 
are  after).  Consider  what  taking  the  variation  with  respect  to  n  does  to  equations  1  and  2; 

oz  Zk  l  2k  k 

(5£:[n(r)]=9?  ^  obsK))' (5) 

m=l 

where  *  is  the  complex  conjugate  and  91  means  taking  the  real  part. 

We  define  the  adjoint  field  by: 

where  (^(r)  is  the  adjoint  field  and  S{r)  is  the  adjoint  source  given  by 

^(^)=ZM'J(//(r-;)-c//„JrJ)*<5(r-rJ  .  (7) 

m=l 

Note  that  here  5  ( r— f^)  is  the  dirac  delta  function  (impulse  function). 

We  can  see  that  (7)  is  such  that  we  can  cast  (5)  as 
5£[n(r)]=91  JJJs(r)5(//(r)rfV 


If  we  substitute  (6)  into  (8)  we  find  that 

space  ^  ^  ^iC  2rC 

We  use  the  following  identities: 

6i(/V\(k=(pV\6ip+V  j^6tp)  (10a) 

and 

.  dip  >  d6(p  Q  . 

.  (10b) 

oz  oz  oz  ^ 

The  second  term  on  the  right  hand  side  of  each  of  these  vanishes  at  the  appropriate 
boundary  of  the  problem  space.  Take  these  terms  out  by  integrating  them  by  parts. 

Substituting  (10)  into  (9)  we  obtain 

6£[n(r)]=-9?JJ/[^^—;^Vl(5(p+^(k^-kJn^)5d']^/rfV  .  (11) 

Space  ^  Z  2fC  2fc 

ik^n 

The  term  in  the  brackets  under  the  integral  is  most  of  (4):  [  ] = — 5  „  . 

k 
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Substituting  for  the  bracketed  term  we  get 

ik^ 

5£[n(r)]=-5RjJJ  -^il)[r)^{f)n{r)5nd\ 

space  ^ 

or 

iB[n(T)]=^3///  V'(T)v.(?)n(T)««d’r  (12) 

^  space 

which,  when  we  compare  it  to  (3)  yields  a  formula  for  the  Frdchet  derivative 

k^ 

V,£[n(r)]=-^3[,p(r)(p(r)n(r)] 

k 

that  we  can  use  in  optimization  techniques  such  as  steepest  descent  or  conjugate  gradient 
methods.  In  our  case  we  have  an  index  of  refraction  that  includes  attenuation,  so  we 
must  make  sure  to  use  a  complex  n  and  dn  in  (12)  which  yields: 
kl 

Vf£[n]=--^[91((//(//)(n,(5n^+n^<5n,.)+3((//t//)(n/n,-n.5n,.)]  (13) 

/c 

where  n.  and  are  the  imaginary  and  real  (attenuation  and  sound  speed)  parts  of  the 

complex  index  of  refraction.  This  is  the  frnal  form  of  the  Frdchet  derivative  that  we  used 
in  our  reconstruction  experiments. 
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